Image processing apparatus, diagnosis supporting method, and recording medium recording image processing program

ABSTRACT

An image processing apparatus includes a processor. The processor receives an observation image of a subject or the observation image and system information, detects a lesioned part candidate from the observation image, estimates a deterioration risk of endoscopy quality from the observation image or the system information, controls a notification form of the lesioned part candidate from an estimation result of the deterioration risk, and notifies the lesioned part candidate according to the control of the notification form.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation application of PCT/JP2019/001712filed on Jan. 21, 2019, the entire contents of which are incorporatedherein by this reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to an image processing apparatus, adiagnosis supporting method, and a recording medium recording an imageprocessing program.

2. Description of the Related Art

Endoscopes have been widely used in a medical field and an industrialfield. For example, in the medical field, a surgeon can find andidentify a lesioned part by viewing an endoscopic image in a subjectdisplayed on a display apparatus and perform treatment on the lesionedpart using a treatment instrument.

There has been generally widely known an image processing apparatus thatapplies a marker such as a frame to and highlights a lesioned partdetected from an endoscopic image in order to prevent a surgeon fromoverlooking the lesioned part when the surgeon views the endoscopicimage.

In endoscopic observation, relative positions of an object in a bodycavity, an image of which is picked up by an endoscope, and an insertionsection of the endoscope inserted into the body cavity can alwayschange. Therefore, it is difficult to correctly detect a once-detectedlesioned part in all frames. Overlooking of a lesioned part candidateregion easily occurs. Accordingly. Japanese Patent Application Laid-OpenPublication No. 2006-255021 proposes a technique for temporally changingdisplay intensity based on a display period in order to prevent theoverlooking of the lesioned part candidate region. Japanese PatentApplication Laid-Open Publication No. 2017-039364 discloses a techniquefor analyzing a sleepiness state of a user and changing a display methodto reduce sleepiness (fatigue) to prevent a driving operation mistake ofthe user.

SUMMARY OF THE INVENTION

An image processing apparatus according to an aspect of the presentinvention includes a processor. The processor: receives an observationimage of a subject or the observation image and system information;detects a lesioned part candidate from the observation image; estimatesa deterioration risk of endoscopy quality from the observation image orthe system information; controls a notification form of the lesionedpart candidate from an estimation result of the deterioration risk; andnotifies the lesioned part candidate according to the control of thenotification form.

A non-transitory computer-readable recording medium recording an imageprocessing program according to an aspect of the present inventionrecords an image processing program for causing a computer to executeprocessing for: receiving an observation image of a subject or theobservation image and system information; detecting a lesioned partcandidate from the observation image; estimating a deterioration risk ofendoscopy quality from the observation image or the system information;controlling a notification form of the lesioned part candidate from anestimation result of the deterioration risk of the endoscopy quality;and notifying the lesioned part candidate according to the control ofthe notification form.

A diagnosis supporting method according to an aspect of the presentinvention includes: detecting a lesioned part candidate from anobservation image of a subject; estimating a deterioration risk ofendoscopy quality from the observation image or system information; andnotifying the lesioned part candidate in a notification formcorresponding to an estimation result of the deterioration risk of theendoscopy quality.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration of a main part of anendoscope system including an endoscope apparatus according to a firstembodiment:

FIG. 2 is a block diagram showing an example of a specific configurationof an image processing apparatus 40 in FIG. 1:

FIG. 3 is a flowchart for explaining the first embodiment:

FIG. 4 is a block diagram showing a second embodiment of the presentinvention;

FIG. 5 is a block diagram showing an example of a specific configurationof a detection-marker-information generating unit 82;

FIG. 6 is a flowchart for explaining operation in a second embodiment;

FIG. 7 is a flowchart for explaining the operation in the secondembodiment:

FIG. 8 is a flowchart for explaining the operation in the secondembodiment;

FIG. 9 is a flowchart for explaining the operation in the secondembodiment;

FIG. 10 is an explanatory diagram for explaining display examples;

FIG. 11 is a block diagram showing a third embodiment of the presentinvention;

FIG. 12 is a flowchart for explaining operation in the third embodiment;

FIG. 13 is a block diagram showing a fourth embodiment of the presentinvention;

FIG. 14 is a flowchart for explaining operation in the fourthembodiment;

FIG. 15 is a block diagram showing a fifth embodiment of the presentinvention;

FIG. 16 is a flowchart for explaining operation in the fifth embodiment;

FIG. 17 is a flowchart for explaining the operation in the fifthembodiment:

FIG. 18 is a block diagram showing a sixth embodiment of the presentinvention:

FIG. 19 is a flowchart for explaining operation in the sixth embodiment;

FIG. 20 is a block diagram showing a seventh embodiment of the presentinvention; and

FIG. 21 is a flowchart for explaining operation in the seventhembodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention are explained in detail below withreference to the drawings.

First Embodiment

FIG. 1 is a diagram showing a configuration of a main part of anendoscope system including an endoscope apparatus according to a firstembodiment. The present embodiment is an embodiment for analyzing ahistory, an endoscopy state, and the like of a user to determine a state(a risk) in which endoscopy quality is likely to be deteriorated, andchanging a display method according to the determination in order toprevent deterioration in the endoscopy quality. Consequently, in thepresent embodiment, for example, it is possible to prevent overlookingof a lesioned part candidate of the user. There is an effect ofimproving the endoscopy quality.

As shown in FIG. 1, an endoscope system 1 includes a light sourcedriving apparatus 11, an endoscope 21, a video processor 31, an imageprocessing apparatus 40, and a display apparatus 95.

The light source driving apparatus 11 includes, for example, a drivecircuit. The light source driving apparatus 11 is connected to theendoscope 21 and the video processor 31. The light source drivingapparatus 11 is configured to generate, based on a light source controlsignal from the video processor 31, a light source driving signal fordriving a light source unit 23 of the endoscope 21 and output thegenerated light source driving signal to the endoscope 21.

The endoscope 21 is connected to the light source driving apparatus 11and the video processor 31. The endoscope 21 includes an insertionsection 22 having an elongated shape insertable into a body cavity of anexaminee. The light source unit 23 and an image pickup unit 24 areprovided at a distal end portion of the insertion section 22.

The light source unit 23 includes a light emitting element such as awhite LED. The light source unit 23 is configured to emit lightaccording to a light source driving signal outputted from the lightsource driving apparatus 11 to generate illumination light and emit thegenerated illumination light to an object such as biological tissue.

The image pickup unit 24 includes an image sensor such as a color CCD ora color CMOS. The image pickup unit 24 is configured to performoperation corresponding to an image pickup control signal outputted fromthe video processor 31. The image pickup unit 24 is configured toreceive reflected light from an object illuminated by the illuminationlight emitted from the light source unit 23, pick up an image of thereceived reflected light and generate an image pickup signal, and outputthe generated image pickup signal to the video processor 31.

The video processor 31 is connected to the light source drivingapparatus 11 and the endoscope 21. The video processor 31 is configuredto generate alight source control signal for controlling a lightemission state of the light source unit 23 and output the light sourcecontrol signal to the light source driving apparatus 11. The videoprocessor 31 is configured to generate and output an image pickupcontrol signal for controlling an image pickup operation of the imagepickup unit 24. The video processor 31 applies predetermined processingto the image pickup signal outputted from the endoscope 21 to generatean observation image of the object. The video processor 31 is configuredto apply highlighting processing and white balance correction processingto the generated observation image and, subsequently, sequentiallyoutput the observation image to the image processing apparatus 40 frameby frame and output the generated observation image to the displayapparatus 95 as an image for display.

The image processing apparatus 40 includes an electronic circuit such asan image processing circuit. The image processing apparatus 40 isconfigured to generate an image for display based on the observationimage outputted from the video processor 31 and perform operation forcausing the display apparatus 95 to display the generated image fordisplay.

The display apparatus 95 includes a monitor or the like and isconfigured to be able to display the observation image from the videoprocessor 31 and to display the image for display outputted from theimage processing apparatus 40.

FIG. 2 is a block diagram showing an example of a specific configurationof the image processing apparatus 40 in FIG. 1.

As shown in FIG. 2, the image processing apparatus 40 includes an inputunit 50, a diagnosis supporting unit 60, a risk estimating unit 70, anotification control unit 80, and a notification output unit 90. Notethat the diagnosis supporting unit 60, the risk estimating unit 70, andthe notification control unit 80 may be configured by a processor usinga CPU or an FPGA, may operate according to a program stored in anot-shown memory to control the respective units, or may realize a partor all of functions with a hardware electronic circuit.

The input unit 50 captures the observation image inputted from the videoprocessor 31 and outputs the observation image to the diagnosissupporting unit 60. The input unit 50 may be configured to capturesystem information from the video processor 31. Note that the systeminformation is included in header information of the observation imageand inputted in some cases and is inputted as data separate from theobservation image in other cases. In the present embodiment, the systeminformation includes a history and an endoscopy state of the user. Theinput unit 50 outputs the inputted observation image to the diagnosissupporting unit 60 and the risk estimating unit 70. The input unit 50 isconfigured to, when the system information is inputted separately fromthe observation image, output the inputted system information to therisk estimating unit 70.

The diagnosis supporting unit 60 is configured to detect, with apublicly-known method, a lesioned part candidate based on the inputtedobservation image. The diagnosis supporting unit 60 outputs informationconcerning the detected lesioned part candidate to the notificationcontrol unit 80.

The notification control unit 80 receives the observation image from theinput unit 50, receives the information concerning the lesioned partcandidate from the diagnosis supporting unit 60, and generates, in theobservation image, information for notifying a detection result of thelesioned part candidate detected by the diagnosis supporting unit 60.The notification control unit 80 outputs the generated information tothe notification output unit 90. The notification output unit 90 isconfigured to notify the detection result of the lesioned part candidateto the user based on the information outputted from the notificationcontrol unit 80. For example, the notification output unit 90 notifiesthe detection result of the lesioned part candidate to the user withnotification by an image, notification by sound, or the like.

For example, when the notification output unit 90 performs thenotification by an image, the notification control unit 80 generatesinformation for displaying an image indicating a position of thelesioned part candidate (hereinafter referred to as detection marker) onthe observation image and outputs the information to the notificationoutput unit 90. The notification output unit 90 generates an image fordisplay obtained by combining, with the observation image received fromthe input unit 50, the image of the detection marker based on theinformation outputted from the notification control unit 80. Thenotification output unit 90 gives the generated image for display to thedisplay apparatus 95 and displays the image for display on a displayscreen 95 a.

In this case, the doctor or the like performs final determination withthe observation image and the detection marker displayed on the displayscreen 95 a of the display apparatus 95 referring to the lesioned partcandidate detected by the diagnosis supporting unit 60. In this case, itis likely that the detection marker is overlooked or attentiveness isreduced according to experience, a fatigue degree, or the like of thedoctor or the like who observes the observation image. Note that thesame problem occurs when the notification is performed by sound. Forexample, depending on magnitude of volume and a notification period,there are problems in that a hearing error occurs and attentiveness isreduced by the sound.

Accordingly, in the present embodiment, the risk estimating unit 70 isprovided in order to optimize a notification form of the notification bythe notification control unit 80 according to the user. The riskestimating unit 70 is configured to estimate, based on the observationimage, a risk of deterioration in endoscopy quality and output anestimation result of the risk to the notification control unit 80. Notethat the risk estimating unit 70 may be configured to estimate the riskof deterioration in the endoscopy quality using the system information.In other words, the risk estimating unit 70 estimates a risk based on atleast one of the observation image or the system information.

For example, the risk estimating unit 70 may use, based on theobservation image and the system information, an analysis result of afatigue degree and an experience level of the user as an estimationresult of the risk of deterioration in the endoscopy quality. The riskestimating unit 70 generates a risk estimation result for changing thenotification form to prevent overlooking.

For example, when the notification is performed by the image, the riskestimating unit 70 estimates the risk of deterioration in the endoscopyquality in order to perform change control for a display form of thedetection marker. The notification control unit 80 optimizes, followingthe risk estimation result, the display form of the detection markeraccording to the user. Note that, in the following explanation, anexample in which the notification is performed using the image isexplained. However, the same control is possible when the notificationis performed by sound.

The notification control unit 80 is configured to receive the riskestimation result of the risk estimating unit 70 and change the displayform of the detection marker. The notification control unit 80 can causethe user to recognize presence or absence of detection of a lesionedpart candidate according to display or non-display of the detectionmarker and cause the user to recognize a position of a lesioned part ina body according to a display position in the observation image. In thisway, the detection marker is displayed in, for example, a display formcorresponding to the risk estimation result, which is the analysisresult of the fatigue degree and the experience level of the user. It ispossible to prevent overlooking and improve endoscopy quality.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 3. FIG. 3 is a flowchart forexplaining the first embodiment.

For example, when the light source driving apparatus 11 and the videoprocessor 31 are turned on, the endoscope 21 emits illumination light toan object, receives reflected light from the object, picks up an imageof the received reflected light and generates an image pickup signal,and outputs the generated image pickup signal to the video processor 31.

The video processor 31 applies predetermined processing to the imagepickup signal outputted from the endoscope 21 to generate an observationimage of the object and sequentially outputs the generated observationimage to the image processing apparatus 40 frame by frame. In otherwords, the input unit 50 acquires an endoscopic image (the observationimage), which is an in-vivo luminal image, from the video processor 31(S1). The system information may be included in header information ofthe observation image. The input unit 50 may be configured to, when thesystem information is not included in the header information of theobservation image, capture the system information separately from theobservation image. Note that the input unit 50 may be configured tocapture only the observation image not including the system information.

Subsequently, in step S2, the diagnosis supporting unit 60 receives theobservation image from the input unit 50, detects a lesioned partcandidate from the observation image, and outputs a detection result tothe notification control unit 80.

In step S3, the risk estimating unit 70 receives at least one of theobservation image or the system information from the input unit 50 andestimates a risk of deterioration in endoscopy quality. The riskestimating unit 70 outputs an estimation result of the risk to thenotification control unit 80. Note that steps S2 and S3 may be executein order of steps S3 and S2 or may be simultaneously executed.

The notification control unit 80 generates information for displaying,on the observation image received from the input unit 50, a detectionmarker for specifying the lesioned part candidate detected by thediagnosis supporting unit 60. In the present embodiment, thenotification control unit 80 generates information for displaying adetection marker in a display form corresponding to the estimationresult of the risk by the risk estimating unit 70 (S4).

The notification output unit 90 displays the detection marker on thedisplay screen 95 a of the display apparatus based on the informationoutputted from the notification control unit 80 (S5). Note that an imageobtained by superimposing the detection marker on the observation imagemay be outputted from the notification output unit 90, or that the imageof the detection marker may be outputted from the notification outputunit 90 and the display apparatus 95 may display the detection marker ina manner superimposed on the observation image from the video processor31.

As explained above, in the present embodiment, the risk of deteriorationin the endoscopy quality is estimated and the display method is changedaccording to a result of the estimation in order to prevent thedeterioration in the endoscopy quality. Consequently, it is possible toimprove the endoscopy quality.

Second Embodiment

FIG. 4 is a block diagram showing a second embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 41 is adopted instead of the image processing apparatus 40.FIG. 4 shows an example of a specific configuration of the imageprocessing apparatus 41. The present embodiment is an example in which arisk of deterioration in endoscopy quality is estimated according to ananalysis of a fatigue degree of a user.

A configuration of the input unit 50 in FIG. 4 is the same as theconfiguration shown in FIG. 2. In the present embodiment, the example isexplained in which a display control unit 81 is adopted as a specificexample of the notification control unit 80 and a display output unit 91is adopted as a specific example of the notification output unit 90.

The input unit 50 outputs an observation image to the diagnosissupporting unit 60 and the display control unit 81. The input unit 50outputs at least one of the observation image or the system informationto the risk estimating unit 70. The display output unit 91 displays animage for display outputted from the display control unit 81 on thedisplay screen 95 a of the display apparatus 95.

In the present embodiment, the diagnosis supporting unit 60 includes alesioned-part-candidate detecting unit 61. The lesioned-part-candidatedetecting unit 61 is configured to detect a lesioned part candidateincluded in the observation image sequentially outputted from the inputunit 50. The lesioned-part-candidate detecting unit 61 detects alesioned part candidate from the observation image by performingprocessing for applying, to the observation image, an imagediscriminator that acquires, in advance, with a learning method such asdeep learning, a function capable of discriminating a lesioned partcandidate. Note that the detection of the lesioned part candidate is notlimited to the learning method described above and other methods may beused. For example, polyp candidate detection processing disclosed inJapanese Patent Application Laid-Open Publication No. 2007-244518 may beused.

The lesioned-part-candidate detecting unit 61 is configured to determinea region on the observation image of the detected lesioned partcandidate (hereinafter referred to as lesioned part candidate region)and output information indicating the lesioned part candidate region tothe display control unit 81 as a detection result of the lesioned partcandidate.

The display control unit 81 includes a detection-marker-informationgenerating unit 82. The detection-marker-information generating unit 82receives the information indicating the lesioned part candidate regionand, in order to cause the user to recognize presence of the lesionedpart candidate detected in the diagnosis supporting unit 60, generates,for example, information for generating an image (a detection marker)surrounding the lesioned part candidate region in the observation imageand outputs the information to the display output unit 91.

The display output unit 91 includes an image combining unit 92. Theimage combining unit 92 generates, based on the information outputtedfrom the display control unit 81, an image for display obtained bysuperimposing the detection marker in the observation image receivedfrom the input unit 50 and outputs the image for display to the displayapparatus 95. Note that the display output unit 91 is also configured tobe able to output the detection marker from thedetection-marker-information generating unit 82 as-is without combiningthe detection marker with the observation image. The display apparatus95 may also be configured to display the detection marker from thedisplay output unit 91 in a manner superimposed in the observation imagefrom the video processor 31.

The detection marker generated by the information of thedetection-marker-information generating unit 82 has a form necessary forenabling the user to visually recognize presence of the lesioned partcandidate. For example, a shape of the detection marker may be aquadrangle, a triangle, a circle, a star shape, or the like or may beany other shapes in some cases. The detection marker may be an image notsurrounding the lesioned part candidate if the detection marker canindicate the presence and a position of the lesioned part candidate.Further, the detection-marker-information generating unit 82 maygenerate a message indicating a lesioned part as support information anddisplay the message in a form of a popup message or the like near thelesioned part to indicate the presence of the lesioned part.

In the present embodiment, the detection-marker-information generatingunit 82 is configured to change the display form of the detection markerbased on the risk estimation result of the risk estimating unit 70. Inother words, the detection-marker-information generating unit 82 changesthe display form to cause the user to easily recognize the presence ofthe lesioned part candidate and prevent the observation image from beingconfirmed as much as possible. In this case, in the present embodiment,the risk estimating unit 70 is configured to control, following the riskestimation result based on an analysis of a fatigue degree of the user,the change of the display form by the detection-marker-informationgenerating unit 82.

The risk estimating unit 70 includes a fatigue-degree analyzing unit 71.The fatigue-degree analyzing unit 71 analyzes the fatigue degree of theuser to obtain an analysis result. The risk estimating unit 70 may usean analysis result of the fatigue-degree analyzing unit 71 as a riskestimation result. In the present embodiment, the fatigue-degreeanalyzing unit 71 is configured by an operation-log analyzing unit 73.The operation-log analyzing unit 73 analyzes an operation log of theendoscope 21 to analyze the fatigue degree of the user. In the presentembodiment, as an example, the operation-log analyzing unit 73 isconfigured by a number-of-times-of-hold-change analyzing unit 73 a and anumber-of-times-of-twisting analyzing unit 73 b.

The number-of-times-of-hold-change analyzing unit 73 a is configured toanalyze, with the operation log of the endoscope 21, the number of timesof hold change of the endoscope 21. For example, thenumber-of-times-of-hold-change analyzing unit 73 a calculates the numberof times the user changes the hold of the insertion section 22 of theendoscope 21 in any period acquired by a predetermined method. In an actof the user changing the hold of the insertion section 22, a hand of theuser is removed from the insertion section 22. Therefore, the actappears as a change such as shaking of the observation image. Forexample, the number-of-times-of-hold-change analyzing unit 73 a maydetect such a change of the observation image with an image analysis forthe observation image to analyze the change of the hold of the insertionsection 22 by the user and acquire the number of times of the holdchange. For example, it is also possible to attach a not-shownacceleration sensor or the like to the insertion section 22 in order toanalyze the number of times of hold change by analyzing an output of theacceleration sensor. The risk estimating unit 70 may acquire such anoutput of the acceleration sensor as the system information to analyzethe number of times of hold change.

Note that information in a period including the any period can beacquired by various methods. The image processing apparatus 40 andrespective image processing apparatuses explained below may acquire aset period from a not-shown device such as a timer or a sensor, mayacquire the set period from information concerning a photographing timeincluded in image information, or may acquire the set period from systeminformation based on user setting.

The number-of-times-of-twisting analyzing unit 73 b is configured toanalyze, with the operation log of the endoscope 21, the number of timesof twisting of the endoscope 21. For example, thenumber-of-times-of-twisting analyzing unit 73 b includes a not-showntimer and calculates the number of times the user twists the insertionsection 22 of the endoscope 21 in any period. In an act of the usertwisting the insertion section 22, the insertion section 22 rotates.Therefore, the act appears as a change in which the observation imagerotates. For example, the number-of-times-of-twisting analyzing unit 73b may detect such a change of the observation image with an imageanalysis for the observation image to analyze the twisting of theinsertion section 22 by the user and acquire the number of times of thetwisting. For example, it is also possible to attach a not-shown gyrosensor or the like to the insertion section 22 in order to analyze thenumber of times of twisting by analyzing an output of the gyro sensor.The risk estimating unit 70 may analyze the number of times of twistingby acquiring such an output of the gyro sensor as the systeminformation.

The fatigue-degree analyzing unit 71 may analyze the fatigue degree ofthe user according to at least one of the number of times of hold changecalculated by the number-of-times-of-hold-change analyzing unit 73 a orthe number of times of twisting calculated by thenumber-of-times-of-twisting analyzing unit 73 b and set an analysisresult as the risk estimation result. For example, the fatigue-degreeanalyzing unit 71 may determine that the fatigue degree is higher as thenumber of times of hold change or the number of times of twisting of theendoscope 21 by the user is larger and estimate that the risk ofdeterioration in the endoscopy quality increases. Conversely, the riskestimating unit 70 may determine that the fatigue degree is lower as thenumber of times of hold change or the number of times of twisting of theendoscope 21 by the user is smaller and estimate that the risk ofdeterioration in the endoscopy quality decreases. The risk estimationresult based on the fatigue degree of the user outputted from the riskestimating unit 70 is supplied to the display control unit 81.

Note that, in the present embodiment, the example is explained in whichthe operation-log analyzing unit 73 is configured by thenumber-of-times-of-hold-change analyzing unit 73 a and thenumber-of-times-of-twisting analyzing unit 73 b. However, theoperation-log analyzing unit 73 may be configured by one of thenumber-of-times-of-hold-change analyzing unit 73 a and thenumber-of-times-of-twisting analyzing unit 73 b.

FIG. 5 is a block diagram showing an example of a specific configurationof the detection-marker-information generating unit 82.

In FIG. 5, an example is shown in which the detection-marker-informationgenerating unit 82 is configured by a display-time control unit 83 and adisplay-content control unit 84. The display-time control unit 83changes a display time of the detection marker based on the inputtedrisk estimation result. For example, the display-time control unit 83may set a display time corresponding to the fatigue degree of the userindicated by the risk estimation result. For example, the display timecan be set longer as the fatigue degree of the user is higher and can beset shorter as the fatigue degree of the user is lower.

The display-content control unit 84 changes content (quality) of thedisplay of the detection marker based on the inputted risk estimationresult. In the example shown in FIG. 5, the display-content control unit84 is configured by a color-tone changing unit 84 a and adisplay-determination-level changing unit 84 b. The color-tone changingunit 84 a changes a color tone of the detection marker based on theinputted risk estimation result. For example, the color-tone changingunit 84 a may set a color tone corresponding to the fatigue degree ofthe user indicated by the risk estimation result. For example, thecolor-tone changing unit 84 a may set the detection marker to a moreconspicuous color tone as the fatigue degree of the user is higher. Forexample, the color-tone changing unit 84 a may set brightness and chromaof the detection marker higher as the fatigue degree of the user ishigher. In this case, the detection marker is sensuously moreconspicuous display as the fatigue degree of the user is higher.Conversely, the detection marker can sensuously have a more naturalcolor as the fatigue degree of the user is lower.

The display-determination-level changing unit 84 b changes adetermination level of display and non-display (hereinafter referred toas display determination level) of the detection marker based on theinputted risk estimation result. Note that the detection marker isexplained as being less easily displayed as the display determinationlevel is lower and being more easily displayed as the displaydetermination level is higher. For example, even if the lesioned partcandidate is the same, the detection marker sometimes is not displayedif the display determination level is low and is sometimes displayed ifthe display determination level is high.

The display-determination-level changing unit 84 b may determine thedisplay determination level based on only the risk estimation result.However, in the example shown in FIG. 5, the display-determination-levelchanging unit 84 b includes a lesion analyzing unit 85 in order todetermine the display determination level. The lesion analyzing unit 85is configured to analyze a nature of the lesioned part candidatedetected by the lesioned-part-candidate detecting unit 61. The lesionanalyzing unit 85 analyzes whether the lesioned part candidate is easilyoverlooked. For example, the lesion analyzing unit 85 receivesinformation such as a shape, a size, and a color of the lesioned partcandidate from the lesioned-part-candidate detecting unit 61 anddetermines a degree of overlooking likelihood.

The display-determination-level changing unit 84 b determines thedisplay determination level based on the fatigue degree of the userindicated by the risk estimation result and the degree of overlookinglikelihood. The display determination level is set to a higher value asthe degree of overlooking likelihood is higher. Thedisplay-determination-level changing unit 84 b is configured to set thedisplay determination level to a higher value as the fatigue degree ofthe user is higher. Therefore, with the control by thedisplay-determination-level changing unit 84 b, the detection marker ismore easily displayed as the lesioned part candidate is more easilyoverlooked and the fatigue degree is higher and the detection marker isless easily displayed as the lesioned part candidate is less easilyoverlooked and the fatigue degree is lower.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 6 to FIG. 10. FIG. 6 to FIG. 9 areflowcharts for explaining operation in the second embodiment. FIG. 10 isan explanatory diagram for explaining display examples.

The operation in the present embodiment is the same as the operationshown in FIG. 3. The present embodiment explains an example of aspecific flow about steps S3 and S4 in FIG. 3.

When the light source driving apparatus 11 and the video processor 31are turned on, the endoscope 21 emits illumination light to an object,receives reflected light from the object, picks up an image of thereceived reflected light and generates an image pickup signal, andoutputs the generated image pickup signal to the video processor 31.

The video processor 31 applies predetermined processing to the imagepickup signal outputted from the endoscope 21 to generate an observationimage of the object and sequentially outputs the generated observationimage to the image processing apparatus 41 frame by frame. The inputunit 50 acquires an endoscopic image (the observation image), which isan in-vivo luminal image, from the video processor 31 (S1 in FIG. 3).Note that system information is sometimes included in the observationimage. The input unit 50 outputs the acquired image to the diagnosissupporting unit 60 and the risk estimating unit 70. Note that when thesystem information is inputted separately from the observation image,the input unit 50 outputs the acquired system information to the riskestimating unit 70.

The lesioned-part-candidate detecting unit 61 detects a lesioned partcandidate from the observation image using a learning method such asdeep learning (S2 in FIG. 3). The lesioned-part-candidate detecting unit61 determines a lesioned part candidate region indicating a range in animage of the detected lesioned part candidate and outputs informationindicating the lesioned part candidate region to the display controlunit 81 as a detection result of the lesioned part candidate.

On the other hand, the fatigue-degree analyzing unit 71 of the riskestimating unit 70 analyzes a fatigue degree of the user in step S31 inFIG. 6. For example, the fatigue-degree analyzing unit 71 analyzes anoperation log of the endoscope 21 with the operation-log analyzing unit73. FIG. 7 shows a specific flow of the operation-log analyzing unit 73.The operation-log analyzing unit 73 analyzes the number of times of holdchange of the endoscope 21 with the number-of-times-of-hold-changeanalyzing unit 73 a (S311). The operation-log analyzing unit 73 analyzesthe number of times of twisting of the endoscope 21 with thenumber-of-times-of-twisting analyzing unit 73 b (S312).

Note that the analysis of the number of times of hold change and theanalysis of the number of times of twisting in steps S311 and S312 maybe performed in reverse order or only one of the analyses may beperformed. The fatigue-degree analyzing unit 71 calculates a fatiguedegree based on an analysis result of at least one of the analyzednumber of times of hold change or the analyzed number of times oftwisting (S313). The fatigue-degree analyzing unit 71 outputs thecalculated fatigue degree to the display control unit 81 as a riskestimation result. Note that the fatigue-degree analyzing unit 71 mayset the analyzed number of times as an analysis result of the fatiguedegree and directly output the analysis result of the fatigue degree asthe risk estimation result. In this case, it is estimated that thefatigue degree is higher and a risk of deterioration in quality ishigher as the analyzed number of times is larger.

The detection-marker-information generating unit 82 of the displaycontrol unit 81 sets a form of the detection marker based on theinputted risk estimation result and generates information for displayingthe detection marker corresponding to the setting. For example, thedetection-marker-information generating unit 82 sets, with thedisplay-time control unit 83, a display time based on the riskestimation result (S41 in FIG. 8). The detection-marker-informationgenerating unit 82 sets, with the display-content control unit 84,display content based on the risk estimation result (S42 in FIG. 8).Note that the settings in step S41 and S42 may be performed in reverseorder or only one of the settings may be performed.

FIG. 9 shows an example of a specific flow of step S42 in FIG. 8. Forexample, the display-content control unit 84 changes, with thecolor-tone changing unit 84 a, a color tone according to the riskestimation result (S421 in FIG. 9). For example, when the riskestimation result indicates that the risk of deterioration in theendoscopy quality is relatively high, the color-tone changing unit 84 asets the detection marker to a conspicuous color tone.

The display-content control unit 84 determines a display determinationlevel with the display-determination-level changing unit 84 b. Forexample, in step S422, the display-determination-level changing unit 84b analyzes a nature of the lesioned part candidate. Thedisplay-determination-level changing unit 84 b determines the displaydetermination level based on the risk estimation result and the natureof the lesioned part candidate (S423). For example, about a lesionedpart candidate having a small size and an easily overlooked nature, thedisplay-determination-level changing unit 84 b sets the displaydetermination level high to make it easy to display the lesioned partcandidate. The display-determination-level changing unit 84 b furtherchanges the display determination level according to the risk estimationresult. For example, when the fatigue degree of the user is relativelyhigh, the display-determination-level changing unit 84 b sets thedisplay determination level higher to make it easy to display thelesioned part candidate.

Note that the settings in step S421 and step S423 may be performed inreverse order or only the setting in step S421 may be performed. Thedisplay-determination-level changing unit 84 b may change apredetermined display determination level according to only the riskestimation result to determine the display determination level.

In step S43, the detection-marker-information generating unit 82generates, based on the setting of at least one of the set display time,the set color tone, or the set display determination level, informationfor displaying the detection marker and outputs the information to thedisplay output unit 91. The image combining unit 92 of the displayoutput unit 91 generates, based on the information outputted from thedetection-marker-information generating unit 82, an image for displayobtained by superimposing the detection marker on the observation imageand gives the image for display to the display apparatus 95.Consequently, the display apparatus 95 displays the observation imagesuperimposed with the detection marker on the display screen 95 a.

FIG. 10 shows examples of images displayed on the display screen 95 a ofthe display apparatus 95. Images 101 a to 101 c shown in FIG. 10indicate, with respect to the same observation image 101, three displayexamples of detection markers in the cases in which risk estimationresults are different. In the observation image 101, a lesioned partcandidate 101 a detected by the diagnosis supporting unit 60 is present.Detection markers 102 to 104 are displayed to surround the lesioned partcandidate 101 a.

For example, the detection marker 102 is a display example in the caseof a medium fatigue degree. The detection marker 103 is a displayexample in the case of a large fatigue degree. The detection marker 104is a display example in the case of an extremely small fatigue degree.On the figure, the detection marker 103 is displayed by a thicker linecompared with the detection marker 102 to indicate that, for example,the detection marker 103 is a display corresponding to a relatively highdisplay determination level in a relatively long display time and arelatively conspicuous color tone.

As an example of the change of the display content, the color tone andthe display determination level are explained. However, as shown in FIG.10, thickness of a line of a frame of a detection marker, a type of theframe, and the like may be changed. For example, the thickness of theline of the frame of the detection marker may be increased as thefatigue degree is higher. Alternatively, the detection marker may be litand flashed according to the risk estimation result. A flashing periodmay be changed according to the risk estimation result. For example, asindicated by the detection marker 104, when the fatigue degree isrelatively low, the detection marker may be displayed by a broken line.

As explained above, in the present embodiment, the risk of deteriorationin the endoscopy quality is estimated by the analysis of the fatiguedegree of the user and the display form of the detection marker ischanged based on the risk estimation result. Consequently, when thefatigue degree of the user is relatively high, it is possible to preventoverlooking of the detection marker and improve the endoscopy qualityby, for example, clearly displaying the detection marker. When thefatigue degree of the user is relatively low, it is possible to improvethe endoscopy quality by displaying the detection marker not todeteriorate visibility of the observation image.

Third Embodiment

FIG. 11 is a block diagram showing a third embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 42 is adopted instead of the image processing apparatus 40.The image processing apparatus 42 shown in FIG. 11 is different from theimage processing apparatus 41 shown in FIG. 4 in that the fatigue-degreeanalyzing unit 71 is configured by an endoscopy-state analyzing unit 74.The other components are the same as the components in the secondembodiment and explanation of the components is omitted. The presentembodiment is an example in which a fatigue degree of a user isestimated by an analysis of an endoscopy state.

The endoscopy-state analyzing unit 74 analyzes a state of an endoscopyto analyze the fatigue degree of the user. In the present embodiment, asan example, the endoscopy-state analyzing unit 74 is configured by anendoscopy-elapsed-time analyzing unit 74 a and anumber-of-times-of-continuous-endoscopy analyzing unit 74 b.

The endoscopy-elapsed-time analyzing unit 74 a is configured to analyzean endoscopy elapsed time. For example, the endoscopy-elapsed-timeanalyzing unit 74 a calculates a time in which the user performs theendoscopy in any period acquired by the various methods explained above.At the endoscopy time, information for specifying a tester such as aname of the tester is inputted. The endoscopy-elapsed-time analyzingunit 74 a is capable of analyzing the endoscopy elapsed time with systeminformation. Note that the information for specifying the tester issometimes included in an observation image separately from the systeminformation. The endoscopy-elapsed-time analyzing unit 74 a is alsocapable of analyzing endoscopy elapsed times of respective users fromthe observation image.

For example, as a freely selected time, for example, one day, one week,one month, and the like can be set. For example, when the endoscopyelapsed times in one day are relatively long about the respective users,fatigue degrees are considered to increase as the elapsed timesincrease. The fatigue-degree analyzing unit 71 can analyze that thefatigue degree is higher as the endoscopy elapsed time calculated by theendoscopy-elapsed-time analyzing unit 74 a is longer. The riskestimating unit 70 can estimate that the risk of deterioration in theendoscopy quality increases.

The number-of-times-of-continuous-endoscopy analyzing unit 74 b analyzesthe numbers of times of continuous endoscopy of the endoscopy about therespective users. The number-of-times-of-continuous-endoscopy analyzingunit 74 b is capable of analyzing the number of times of continuousendoscopy for each of the users with the system information. Thenumber-of-times-of-continuous-endoscopy analyzing unit 74 b is alsocapable of analyzing the numbers of times of continuous endoscopy of therespective users from the observation image.

For example, it is assumed that a plurality of times of the endoscopyare performed in one day. The fatigue degree is different between whenthe same user continuously performs the endoscopy and when a pluralityof users perform the endoscopy while taking turns. The fatigue degree isconsidered to increase as the number of times the same user continuouslyperforms the endoscopy is larger. The fatigue-degree analyzing unit 71can analyze that the fatigue degree is higher as the number of times ofcontinuous endoscopy calculated by thenumber-of-times-of-continuous-endoscopy analyzing unit 74 b is larger.The risk estimating unit 70 can estimate that the risk of deteriorationin the endoscopy quality increases.

Note that, in the present embodiment, the example is explained in whichthe endoscopy-state analyzing unit 74 is configured by theendoscopy-elapsed-time analyzing unit 74 a and thenumber-of-times-of-continuous-endoscopy analyzing unit 74 b. However,the endoscopy-state analyzing unit 74 may be configured by one of theendoscopy-elapsed-time analyzing unit 74 a and thenumber-of-times-of-continuous-endoscopy analyzing unit 74 b.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 12. FIG. 12 is a flowchart forexplaining operation in the third embodiment.

The operation in the present embodiment is the same as the operationshown in FIG. 3 and FIG. 6. FIG. 12 shows an example of a specific flowdifferent from the flow shown in FIG. 7 about step S31 in FIG. 6.

The fatigue-degree analyzing unit 71 of the risk estimating unit 70analyzes the fatigue degree of the user in step S31 in FIG. 6. Forexample, the fatigue-degree analyzing unit 71 analyzes, with theendoscopy-state analyzing unit 74, endoscopy states of the respectiveusers. FIG. 12 shows an example of specific processing of theendoscopy-state analyzing unit 74. The endoscopy-state analyzing unit 74analyzes an endoscopy elapsed time of the endoscopy with theendoscopy-elapsed-time analyzing unit 74 a (S314). The endoscopy-stateanalyzing unit 74 analyzes the number of times of continuous endoscopyof the endoscopy with the number-of-times-of-continuous-endoscopyanalyzing unit 74 b (S315).

The analyses in steps S314 and S315 may be performed in reverse order oronly one of the analyses may be performed. The fatigue-degree analyzingunit 71 calculates a fatigue degree based on an analysis result of atleast one of the analyzed endoscopy elapsed time or the analyzed numberof times of continuous endoscopy (S316). The fatigue-degree analyzingunit 71 outputs the calculated fatigue degree to the display controlunit 81 as a risk estimation result. Note that the fatigue-degreeanalyzing unit 71 estimates that the fatigue degree is higher and therisk of deterioration in the quality is higher as the endoscopy elapsedtime is longer and as the number of times of continuous endoscopy islarger.

As in the second embodiment, the detection-marker-information generatingunit 82 of the display control unit 81 generates information fordisplaying a detection marker in a form corresponding to the riskestimation result. As a result, a detection marker in a formcorresponding to the fatigue degree of the user is displayed on thedisplay screen 95 a of the display apparatus 95.

The other actions are the same as the actions in the second embodiment.In this way, in the present embodiment, the same effects as the effectsin the second embodiment can be obtained.

Fourth Embodiment

FIG. 13 is a block diagram showing a fourth embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 43 is adopted instead of the image processing apparatus 40.The image processing apparatus 43 shown in FIG. 13 is different from theimage processing apparatus 41 shown in FIG. 4 in that the fatigue-degreeanalyzing unit 71 is configured by a biological-information analyzingunit 75. The other components are the same as the components in thesecond embodiment and explanation of the components is omitted. Thepresent embodiment is an example in which a fatigue degree of a user isestimated by an analysis of biological information.

The biological-information analyzing unit 75 analyzes the biologicalinformation to analyze the fatigue degree of the user. In the presentembodiment, as an example, the biological-information analyzing unit 75is configured by a number-of-times-of-blinking analyzing unit 75 a and apulse-and-the-like analyzing unit 75 b.

The number-of-times-of-blinking analyzing unit 75 a is configured toanalyze the number of times of blinking of the user. In an endoscopyroom or the like, an operating field camera and the like are provided.Image pickup of a face part of the user with the camera is possible.Images from the cameras are also inputted to the video processor 31. Thevideo processor 31 can calculate, with an image analysis of a picked-upimage of the user, the number of times the user blinks. The videoprocessor 31 can output the number of times of blinking of the user tothe image processing apparatus 43 as system information. Thenumber-of-times-of-blinking analyzing unit 75 a analyzes the number oftimes of blinking of the user with the inputted system information.

Note that although an example in which information concerning the numberof times of blinking is acquired from the image is explained above, theinformation may be acquired from devices such as various sensors otherthan the image or may be acquired from the system information.

Note that the risk estimating unit 70 may be configured to capturepicked-up images of the user picked up by the operating field camera andthe like. In this case, the number-of-times-of-blinking analyzing unit75 a is capable of calculating the number of times of blinking of theuser with an image analysis for the captured picked-up images of theuser.

The fatigue degree of the user is considered to be larger as the numberof times of blinking is larger. The fatigue-degree analyzing unit 71 cananalyze that the fatigue degree is higher as the number of times ofblinking calculated by the number-of-times-of-blinking analyzing unit 75a is larger. The risk estimating unit 70 can estimate that the risk ofdeterioration in the endoscopy quality increases.

The pulse-and-the-like analyzing unit 75 b analyzes pulses, bodytemperatures, saturated oxygen amounts, and the like about respectiveusers. The pulse-and-the-like analyzing unit 75 b is capable ofperforming these analyses for each of the users with the systeminformation. The fatigue-degree analyzing unit 71 analyzes the fatiguedegree of the users with analysis results of the pulses, the bodytemperatures, the saturated oxygen amounts, and the like.

Note that, in the present embodiment, the example is explained in whichthe biological-information analyzing unit 75 is configured by thenumber-of-times-of-blinking analyzing unit 75 a and thepulse-and-the-like analyzing unit 75 b. However, thebiological-information analyzing unit 75 may be configured by one of thenumber-of-times-of-blinking analyzing unit 75 a and thepulse-and-the-like analyzing unit 75 b.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 14. FIG. 14 is a flowchart forexplaining operation in the fourth embodiment.

The operation in the present embodiment is the same as the operationshown in FIG. 3 and FIG. 6. FIG. 14 shows an example of a specific flowdifferent from the flow shown in FIG. 7 about step S31 in FIG. 6.

The fatigue-degree analyzing unit 71 of the risk estimating unit 70analyzes the fatigue degree of the user in step S31 in FIG. 6. Forexample, the fatigue-degree analyzing unit 71 analyzes biologicalinformation of the respective users with the biological-informationanalyzing unit 75. FIG. 14 shows an example of specific processing ofthe biological-information analyzing unit 75. The biological-informationanalyzing unit 75 analyzes the number of times of blinking with thenumber-of-times-of-blinking analyzing unit 75 a (S317). Theendoscopy-state analyzing unit 74 analyzes a pulse, a body temperature,a saturated oxygen amount, and the like of the user with thepulse-and-the-like analyzing unit 75 b (S318).

Note that the analyses of steps S317 and S318 may be performed inreverse order or only one of the analyses may be performed. Thefatigue-degree analyzing unit 71 calculates a fatigue degree based on ananalysis result of at least one of the analyzed number of times ofblinking, the analyzed pulse, or the like (S319). The fatigue-degreeanalyzing unit 71 outputs the calculated fatigue degree to the displaycontrol unit 81 as the risk estimation result. Note that thefatigue-degree analyzing unit 71 estimates that the fatigue degree ishigher and the risk of deterioration in the endoscopy quality is higheras the number of times of blinking is larger. The fatigue-degreeanalyzing unit 71 may calculate the fatigue degree based on changes ofthe pulse, the body temperature, and the saturated oxygen amount of theuser during an endoscopy or may store information at normal time of eachof the users and calculate the fatigue degree by comparing the pulse,the body temperature, and the saturated oxygen amount of the user duringan endoscopy with the stored information.

As in the second embodiment, the detection-marker-information generatingunit 82 of the display control unit 81 generates information fordisplaying a detection marker in a form corresponding to the riskestimation result. As a result, a detection marker in a formcorresponding to the fatigue degree of the user is displayed on thedisplay screen 95 a of the display apparatus 95.

The other actions are the same as the actions in the second embodiment.In this way, in the present embodiment as well, the same effects as theeffects in the second embodiment can be obtained.

Note that, as the risk estimating unit 70, the fatigue-degree analyzingunit 71 including at least one of the number-of-times-of-hold-changeanalyzing unit 73 a, the number-of-times-of-twisting analyzing unit 73b, the endoscopy-elapsed-time analyzing unit 74 a, thenumber-of-times-of-continuous-endoscopy analyzing unit 74 b, thenumber-of-times-of-blinking analyzing unit 75 a, or thepulse-and-the-like analyzing unit 75 b may be adopted to analyze thefatigue degree of the user.

Fifth Embodiment

FIG. 15 is a block diagram showing a fifth embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 44 is adopted instead of the image processing apparatus 40.FIG. 15 shows an example of a specific configuration of the imageprocessing apparatus 44. The image processing apparatus 44 shown in FIG.15 is different from the image processing apparatus 41 shown in FIG. 4in that the risk estimating unit 70 adopts an experience-level analyzingunit 72 instead of the fatigue-degree analyzing unit 71. The othercomponents are the same as the components in the second embodiment andexplanation of the components is omitted. The present embodiment is anexample in which a risk of deterioration in endoscopy quality isestimated by an analysis of an experience level of a user.

The risk estimating unit 70 includes the experience-level analyzing unit72. The experience-level analyzing unit 72 analyzes the experience levelof the user to obtain an analysis result. The risk estimating unit 70may use the analysis result of the experience-level analyzing unit 72 asa risk estimation result. In the present embodiment, theexperience-level analyzing unit 72 is configured by an operation-loganalyzing unit 76. The operation-log analyzing unit 76 analyzes anoperation log of the endoscope 21 to analyze the experience level of theuser. In the present embodiment, as an example, the operation-loganalyzing unit 76 is configured by anumber-of-times-of-insertion/removal analyzing unit 76 a and areaching-time analyzing unit 76 b.

The number-of-times-of-insertion/removal analyzing unit 76 a isconfigured to analyze the number of times of insertion/removal of theendoscope 21 with the operation log of the endoscope 21. For example,the number-of-times-of-insertion/removal analyzing unit 76 a calculatesthe number of times of insertion/removal of the insertion section 22from when the insertion section 22 starts to be inserted into a bodyuntil when the insertion section 22 reaches an endoscopy target part.According to an act of the user inserting and removing the insertionsection 22, the image pickup unit 24 provided in the insertion section22 also moves and an observation image changes. For example, thenumber-of-times-of-insertion/removal analyzing unit 76 a may beconfigured to detect such a change of the observation image with animage analysis for the observation image to analyze the insertion andthe removal of the insertion section 22 by the user and acquire thenumber of times of the insertion and the removal. For example, it isalso possible to attach a not-shown acceleration sensor or the like tothe insertion section 22 in order to analyze the number of times ofinsertion/removal by analyzing an output of the acceleration sensor. Therisk estimating unit 70 may acquire such an output of the accelerationsensor as system information to analyze the number of times ofinsertion/removal.

The reaching-time analyzing unit 76 b is configured to analyze, with theoperation log of the endoscope 21, a time period (a reaching time)required for the insertion section 22 to reach the endoscopy targetpart. In other words, the reaching-time analyzing unit 76 b calculates atime period from when the insertion section 22 starts to be insertedinto the body until when the insertion section 22 reaches the endoscopytarget part. It is possible to detect an insertion start time and anendoscopy target part reaching time of the insertion section 22 with animage analysis of the observation image and clocking by a timer or thelike. The reaching-time analyzing unit 76 b may be configured to analyzethe reaching time with the image analysis of the observation image. Forexample, it is also possible to transmit the insertion start time andthe endoscopy target part reaching time to the video processor 31 withoperation of an endoscope switch by the user. The risk estimating unit70 may acquire these time points as the system information to analyzethe reaching time.

The experience-level analyzing unit 72 may analyze the experience levelof the user according to at least one of the number of times ofinsertion/removal calculated by the number-of-times-of-insertion/removalanalyzing unit 76 a or the reaching time calculated by the reaching-timeanalyzing unit 76 b and set an analysis result as the risk estimationresult. For example, the experience-level analyzing unit 72 maydetermine that proficiency is higher, that is, the experience level ishigher as the number of times of insertion/removal of the insertionsection 22 by the user is smaller or the reaching time is shorter andestimate that the risk of deterioration in the endoscopy quality islower. Conversely, the experience-level analyzing unit 72 may determinethat the proficiency is lower, that is, the experience level is lower asthe number of times of insertion/removal of the insertion section 22 bythe user is larger or the reaching time is longer and estimate that therisk of deterioration in the endoscopy quality is higher. The riskestimation result based on the experience level of the user from therisk estimating unit 70 is supplied to the display control unit 81.

Note that, in the present embodiment, the example is explained in whichthe operation-log analyzing unit 76 is configured by thenumber-of-times-of-insertion/removal analyzing unit 76 a and thereaching-time analyzing unit 76 b. However, the operation-log analyzingunit 76 may be configured by one of thenumber-of-times-of-insertion/removal analyzing unit 76 a and thereaching-time analyzing unit 76 b.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 16 and FIG. 17. FIG. 16 and FIG. 17are flowcharts for explaining operation in the fifth embodiment.

The operation in the present embodiment is the same as the operationshown in FIG. 3. FIG. 16 shows an example of a specific flow about stepS3 in FIG. 3. FIG. 17 shows an example of a specific flow of step S51 inFIG. 16.

Acquisition of an observation image, detection of a lesioned partcandidate, and display control are the same as the acquisition of anobservation image, the detection of a lesioned part candidate and thedisplay control in the second embodiment. The present embodiment isdifferent from the second embodiment in a method of risk estimation.

The experience-level analyzing unit 72 of the risk estimating unit 70analyzes the experience level of the user in step S51 in FIG. 16. Forexample, the experience-level analyzing unit 72 analyzes the operationlog of the endoscope 21 with the operation-log analyzing unit 76. FIG.17 shows an example of specific processing of the operation-loganalyzing unit 76. The operation-log analyzing unit 76 analyzes thenumber of times of insertion/removal of the insertion section 22 withthe number-of-times-of-insertion/removal analyzing unit 76 a (S511). Theoperation-log analyzing unit 76 analyzes, with the reaching-timeanalyzing unit 76 b, a reaching time until the insertion section 22reaches the endoscopy target part (S512).

Note that the analysis of the number of times of insertion/removal andthe analysis of the reaching time in steps S511 and S512 may beperformed in reverse order or only one of the analyses may be performed.The experience-level analyzing unit 72 calculates an experience level ofthe user based on an analysis result of at least one of the analyzednumber of times of insertion/removal or the analyzed reaching time(S513). The experience-level analyzing unit 72 outputs the calculatedexperience level to the display control unit 81 as the risk estimationresult. Note that the experience-level analyzing unit 72 may setnumerical values of the number of times of insertion/removal and thereaching time as an analysis result of the experience level and directlyoutput the analysis result of the experience level as the riskestimation result. In this case, it is estimated that the proficiency islower (the experience level is lower) and the risk of deterioration inthe quality is higher as the number of times of insertion/removal islarger and the reaching time is longer.

The detection-marker-information generating unit 82 of the displaycontrol unit 81 sets a form of the detection marker based on theinputted risk estimation result and generates, according to the setting,information for displaying the detection marker. For example, thedetection-marker-information generating unit 82 sets, with thedisplay-time control unit 83, a display time based on the riskestimation result (S41 in FIG. 8) and sets, with the display-contentcontrol unit 84, display content based on the risk estimation result(S42 in FIG. 8).

In this way, the detection marker is displayed in the display formaccording to the risk estimation result.

For example, when the number of times of insertion/removal is large orthe reaching time is long, the detection marker is displayed for arelatively long time in a more conspicuous color tone based on the riskestimation result. Consequently, about a user having a low experiencelevel, it is possible to prevent overlooking of the detection markerand, as a result, achieve improvement of the endoscopy quality.

Conversely, when the number of times of insertion/removal is small orthe reaching time is short, the detection marker is displayed for arelatively short time in a more natural color tone based on the riskestimation result. Consequently, about a user having a high experiencelevel, it is possible to prevent visibility of the observation imagefrom being deteriorated by the detection marker and, as a result,achieve improvement of the endoscopy quality.

As explained above, in the present embodiment, the risk of deteriorationin the endoscopy quality is estimated by the analysis of the experiencelevel of the user. The display form of the detection marker is changedbased on the risk estimation result. Consequently, when the experiencelevel of the user is relatively low, it is possible to preventoverlooking of the detection marker and improve the endoscopy qualityby, for example, clearly displaying the detection marker. When theexperience level of the user is relatively high, it is possible toimprove the endoscopy quality by displaying the detection marker suchthat the visibility of the observation image is not deteriorated.

Sixth Embodiment

FIG. 18 is a block diagram showing a sixth embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 45 is adopted instead of the image processing apparatus 40.The image processing apparatus 45 shown in FIG. 18 is different from theimage processing apparatus 44 shown in FIG. 15 in that theexperience-level analyzing unit 72 is configured by an endoscopy-historyanalyzing unit 77. The other components are the same as the componentsin the fifth embodiment and explanation of the components is omitted.The present embodiment is an example in which an experience level of auser is estimated by an analysis of an endoscopy history.

The endoscopy-history analyzing unit 77 analyzes a history of anendoscopy to analyze the experience level of the user. In the presentembodiment, as an example, the endoscopy-history analyzing unit 77 isconfigured by a total-number-of-endoscopies analyzing unit 77 a and atotal-endoscopy-time analyzing unit 77 b.

The total-number-of-endoscopies analyzing unit 77 a is configured toanalyze a total number of endoscopies. For example, thetotal-number-of-endoscopies analyzing unit 77 a calculates the number oftimes the user performs the endoscopy in any period acquired by thevarious methods explained above. At the endoscopy time, information forspecifying a tester such as a name of the tester is inputted. Thetotal-number-of-endoscopies analyzing unit 77 a is capable of analyzingthe total number of endoscopies with system information. Note that theinformation for specifying the tester is sometimes included in anobservation image separately from the system information. Thetotal-number-of-endoscopies analyzing unit 77 a is also capable ofanalyzing the numbers of endoscopies of respective users from theobservation image.

The total-endoscopy-time analyzing unit 77 b is configured to analyze atotal endoscopy time. For example, the total-endoscopy-time analyzingunit 77 b calculates a time period in which the user performs theendoscopy in any period acquired by the various methods explained above.The total-endoscopy-time analyzing unit 77 b is capable of analyzing thetotal endoscopy time with the system information. Note that thetotal-endoscopy-time analyzing unit 77 b is also capable of analyzingtotal endoscopy times of the respective users from the observationimage.

For example, as a freely selected period, for example, one day, oneweek, and one month can be set. For example, when total numbers ofendoscopies in a year are relatively large and the total endoscopy timesare relatively long about the respective users, the proficiency (theexperience level) is considered to be high. The experience-levelanalyzing unit 72 can analyze that the experience level is higher as thetotal number of endoscopies calculated by thetotal-number-of-endoscopies analyzing unit 77 a is larger and the totalendoscopy time calculated by the total-endoscopy-time analyzing unit 77b is longer. The risk estimating unit 70 can estimate that the risk ofdeterioration in the endoscopy quality is small.

Note that, in the present embodiment, the example is explained in whichthe endoscopy-history analyzing unit 77 is configured by thetotal-number-of-endoscopies analyzing unit 77 a and thetotal-endoscopy-time analyzing unit 77 b. However, the endoscopy-historyanalyzing unit 77 may be configured by one of thetotal-number-of-endoscopies analyzing unit 77 a and thetotal-endoscopy-time analyzing unit 77 b.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 19. FIG. 19 is a flowchart forexplaining operation in the sixth embodiment.

The operation in the present embodiment is the same as the operationshown in FIG. 3 and FIG. 16. FIG. 19 shows an example of a specific flowdifferent from the flow shown in FIG. 17 about step S51 in FIG. 16.

The experience-level analyzing unit 72 of the risk estimating unit 70analyzes the experience level of the user in step S51 in FIG. 16. Forexample, the experience-level analyzing unit 72 analyzes endoscopyhistories of respective users with the endoscopy-history analyzing unit77. FIG. 19 shows an example of specific processing of theendoscopy-history analyzing unit 77. The endoscopy-history analyzingunit 77 analyzes a total number of endoscopies with thetotal-number-of-endoscopies analyzing unit 77 a (S514). Theendoscopy-history analyzing unit 77 analyzes a total endoscopy time ofthe endoscopy with the total-endoscopy-time analyzing unit 77 b (S515).

Note that the analyses in steps S514 and S515 may be performed inreverse order or only one of the analyses may be performed. Theexperience-level analyzing unit 72 calculates an experience level basedon an analysis result of at least one of the analyzed total number ofendoscopies or the analyzed total endoscopy time (S516). Theexperience-level analyzing unit 72 outputs the calculated experiencelevel to the display control unit 81 as a risk estimation result. Notethat the experience-level analyzing unit 72 estimates that theexperience level is higher and the risk of deterioration in theendoscopy quality is lower as the total number of endoscopies is largerant the total endoscopy time is longer and estimates that the experiencelevel is lower and the risk of deterioration in the quality is higher asthe total number of endoscopies is smaller and the total endoscopy timeis shorter.

As in the fifth embodiment, the detection-marker-information generatingunit 82 of the display control unit 81 generates information fordisplaying a detection marker in a form corresponding to the riskestimation result. As a result, a detection marker in a formcorresponding to the experience level of the user is displayed on thedisplay screen 95 a of the display apparatus 95.

The other actions are the same as the actions in the fifth embodiment.In this way, in the present embodiment, the same effects as the effectsin the fifth embodiment can be obtained.

Seventh Embodiment

FIG. 20 is a block diagram showing a seventh embodiment of the presentinvention. An endoscope system in the present embodiment is differentfrom the endoscope system 1 shown in FIG. 1 in that an image processingapparatus 46 is adopted instead of the image processing apparatus 40.The image processing apparatus 46 shown in FIG. 20 is different from theimage processing apparatus 44 shown in FIG. 15 in that theexperience-level analyzing unit 72 is configured by a comparing andanalyzing unit 78. The other components are the same as the componentsin the fifth embodiment and explanation of the components is omitted.The present embodiment is an example in which an experience level of auser is estimated by an analysis by comparison with a computer aideddiagnosis (CAD) apparatus.

The comparing and analyzing unit 78 compares a diagnosis result of theuser and a diagnosis result by the CAD for a lesioned part candidate andanalyzes transition of a comparison result. In the present embodiment,as an example, the comparing and analyzing unit 78 is configured by atransition-from-past analyzing unit 78 a and an intra-case transitionanalyzing unit 78 b.

The transition-from-past analyzing unit 78 a compares the diagnosisresult of the user and the diagnosis result of the CAD for the lesionedpart candidate and analyzes transition indicating how the diagnosisresult of the user changes based on the diagnosis result of the CAD.Usually, a degree of correctness of diagnosis for the lesioned partcandidate is considered to be in order of an expert doctor havingextremely high proficiency≥CAD>an inexperienced doctor. Accordingly, itis considered that a difference between a diagnosis result by theinexperienced doctor and a diagnosis result by the CAD is relativelylarge and the difference gradually decreases as the doctor accumulatesexperiences.

Therefore, it is possible to perform comparison of the diagnosis resultof the user and the diagnosis result by the CAD, store a comparisonresult in a not-shown memory, and determine an experience level of theuser according to transition of the comparison result. Thetransition-from-past analyzing unit 78 a performs recording and readoutof such diagnosis results and analyzes the transition of the comparisonresult. The experience-level analyzing unit 72 determines an experiencelevel of the user based on an analysis result. Note that the riskestimating unit 70 can acquire the diagnosis result of the user and thediagnosis result of the CAD based on the system information.

The intra-case transition analyzing unit 78 b compares a diagnosisresult of the user and a diagnosis result by the CAD for a lesioned partcandidate in one case and analyzes transition indicating how thediagnosis result of the user changes based on the diagnosis result ofthe CAD. At a diagnosis time in one case, when the user accumulatesexperiences, a difference between the diagnosis result of the user andthe diagnosis result by the CAD sometimes decreases. The intra-casetransition analyzing unit 78 b performs recording and readout ofdiagnosis results in respective cases and analyzes transition of acomparison result. The experience-level analyzing unit 72 determines anexperience level of the user based on an analysis result.

Note that, in the present embodiment, the example is explained in whichthe comparing and analyzing unit 78 is configured by thetransition-from-past analyzing unit 78 a and the intra-case transitionanalyzing unit 78 b. However, the comparing and analyzing unit 78 may beconfigured by one of the transition-from-past analyzing unit 78 a andthe intra-case transition analyzing unit 78 b.

Subsequently, operation in the embodiment configured as explained aboveis explained with reference to FIG. 21. FIG. 21 is a flowchart forexplaining operation in the seventh embodiment.

The operation in the present embodiment is the same as the operationshown in FIG. 3 and FIG. 16. FIG. 21 shows an example of a specific flowdifferent from FIG. 17 about step S5 l in FIG. 16.

The experience-level analyzing unit 72 of the risk estimating unit 70analyzes an experience level of the user in step S51 in FIG. 16. Theexperience-level analyzing unit 72 analyzes the experience level withthe comparing and analyzing unit 78. FIG. 21 shows an example ofspecific processing of the comparing and analyzing unit 78. Thecomparing and analyzing unit 78 acquires diagnosis results of respectiveusers and a diagnosis result of the CAD (S517 in FIG. 21) and analyzestransition of a difference between a diagnosis result of a user and thediagnosis result of the CAD with the transition-from-past analyzing unit78 a (S518). The experience-level analyzing unit 72 analyzes transitionof the difference between the diagnosis result of the user and thediagnosis result of the CAD about one case with the intra-casetransition analyzing unit 78 b (S519).

Note that the analyses in steps S518 and S519 may be performed inreverse order or only one of the analyses may be performed. Theexperience-level analyzing unit 72 calculates an experience level basedon at least one analysis result of the analysis result of the analyzedtransition from the past and the analysis result of the analyzedtransition in one case (S520). For example, the experience-levelanalyzing unit 72 determines a degree of correctness of the diagnosisresult of the user and calculates an experience level of the useraccording to the transition of the difference between the diagnosisresult of the user and the diagnosis result of the CAD. For example,when the difference between the diagnosis result of the user and thediagnosis result of the CAD decreases, it can be determined that theexperience level of the user increases.

The risk estimating unit 70 outputs the calculated experience level tothe display control unit 81 as a risk estimation result. As in the fifthembodiment, the detection-marker-information generating unit 82 of thedisplay control unit 81 generates information for displaying a detectionmarker in a form corresponding to the risk estimation result. As aresult, a detection marker in a form corresponding to the experiencelevel of the user is displayed on the display screen 95 a of the displayapparatus 95.

The other actions are the same as the actions in the fifth embodiment.In this way, in the present embodiment, the same effects as the effectsin the firth embodiment can be obtained.

Note that, as the risk estimating unit 70, the experience-levelanalyzing unit 72 including at least one of thenumber-of-times-of-insertion/removal analyzing unit 76 a, thereaching-time analyzing unit 76 b, the total-number-of-endoscopiesanalyzing unit 77 a, the total-endoscopy-time analyzing unit 77 b, thetransition-from-past analyzing unit 78 a, or the intra-case transitionanalyzing unit 78 b may be adopted to analyze the experience level ofthe user.

Further, the risk estimating unit 70 may include at least one of thefatigue-degree analyzing unit 71 or the experience-level analyzing unit72 in the second to seventh embodiments.

In the respective embodiments, when the notification is performed bysound or the like, it is also possible to set a notification formcorresponding to the risk estimation result.

Note that, among the techniques explained in the specification, thecontrol mainly explained in the flowcharts can often be set by a programand is sometimes stored in a recording medium or a recording unit. As amethod of recording in the recording medium or the recording unit, thecontrol may be recorded at a product shipment time, may be recordedusing a distributed recording medium, or may be downloaded via theInternet.

The execution order of the steps in the flowcharts may be changed, aplurality of the steps may be simultaneously executed, or the steps maybe executed in different order in every execution unless contrary tonatures of the steps.

Note that, in the embodiments, the portion described as “unit” may beconfigured by a dedicated circuit or may be configured by combining aplurality of general-purpose circuits or may be configured by combining,according to necessity, processors such as a microcomputer and a CPUthat perform operation according to software programmed in advance orsequencers such as an FPGA.

The present invention is not limited to the respective embodiments perse. In an implementation stage, the constituent elements can be modifiedand embodied in a range not departing from the gist of the presentinvention. Various inventions can be formed by appropriate combinationsof a plurality of constituent elements disclosed in the respectiveembodiments. For example, several constituent elements among all theconstituent elements explained in the embodiments may be deleted.Further, the constituent elements in different embodiments may becombined as appropriate.

What is claimed is:
 1. An image processing apparatus comprising aprocessor, wherein the processor: receives an observation image of asubject or the observation image and system information; detects alesioned part candidate from the observation image; estimates adeterioration risk of endoscopy quality from the observation image orthe system information; controls a notification form of the lesionedpart candidate from an estimation result of the deterioration risk; andnotifies the lesioned part candidate according to the control of thenotification form.
 2. The image processing apparatus according to claim1, wherein the processor analyzes a fatigue degree of a user andcalculates an estimation result of the deterioration risk based on thefatigue degree obtained as a result of the analysis of the fatiguedegree.
 3. The image processing apparatus according to claim 2, whereinthe processor analyzes an operation log and calculates the fatiguedegree based on the operation log obtained as a result of the analysisof the operation log.
 4. The image processing apparatus according toclaim 3, wherein the processor analyzes a number of times of hold changeof an endoscope in any period and calculates the fatigue degree based onthe number of times of hold change obtained as a result of the analysisof the number of times of hold change.
 5. The image processing apparatusaccording to claim 3, wherein the processor analyzes a number of timesof twisting of an endoscope in any period and calculates the fatiguedegree based on the number of times of twisting obtained as a result ofthe analysis of the number of times of twisting.
 6. The image processingapparatus according to claim 2, wherein the processor analyzes anendoscopy state log and calculates the fatigue degree based on anendoscopy state obtained as a result of the analysis of the endoscopystate log.
 7. The image processing apparatus according to claim 6,wherein the processor analyzes an endoscopy elapsed time in any periodand calculates the fatigue degree based on the endoscopy elapsed timeobtained as a result of the analysis of the endoscopy elapsed time. 8.The image processing apparatus according to claim 6, wherein theprocessor analyzes a number of times of continuous endoscopy andcalculates the fatigue degree based on the number of times of continuousendoscopy obtained as a result of the analysis of the number of times ofcontinuous endoscopy.
 9. The image processing apparatus according toclaim 2, wherein the processor analyzes biological information andcalculates the fatigue degree based on the biological informationobtained as a result of the analysis of the biological information. 10.The image processing apparatus according to claim 9, wherein theprocessor analyzes a number of times of blinking and calculates thefatigue degree based on the number of times of blinking obtained as aresult of the analysis of the number of times of blinking.
 11. The imageprocessing apparatus according to claim 1, wherein the processoranalyzes an experience level of a user and calculates the riskestimation result based on the experience level obtained as a result ofthe analysis of the experience level.
 12. The image processing apparatusaccording to claim 11, wherein the processor analyzes an operation logand calculates the experience level based on the operation log obtainedas a result of the analysis of the operation log.
 13. The imageprocessing apparatus according to claim 12, wherein the processoranalyzes a number of times of insertion/removal of an endoscope to anendoscopy target part of the endoscope and calculates the experiencelevel based on the number of times of insertion/removal obtained as aresult of the analysis of the number of times of insertion/removal. 14.The image processing apparatus according to claim 12, wherein theprocessor analyzes a reaching time to an endoscopy target part of anendoscope and calculates the experience level based on the reaching timeobtained as a result of the analysis of the reaching time.
 15. The imageprocessing apparatus according to claim 11, wherein the processoranalyzes an endoscopy history of the user and calculates the experiencelevel based on the endoscopy history obtained as a result of theanalysis of the endoscopy history.
 16. The image processing apparatusaccording to claim 15, wherein the processor analyzes at least one of atotal number of endoscopies or a total endoscopy time of an endoscopy inany period to calculate the experience level.
 17. The image processingapparatus according to claim 11, wherein the processor analyzestransition of a comparison result of a diagnosis result of the user anda diagnosis result of a computer aided diagnosis apparatus andcalculates the experience level based on transition of the comparisonresult obtained as a result of the analysis of the transition of thecomparison result.
 18. The image processing apparatus according to claim17, wherein the processor calculates the experience level based on atleast one of transition of a comparison result from past to present ortransition of a comparison result in one endoscopy.
 19. The imageprocessing apparatus according to claim 1, wherein the processor changesa notification time of the lesioned part candidate based on theestimation result of the deterioration risk.
 20. The image processingapparatus according to claim 1, wherein the processor changesnotification content of the lesioned part candidate based on theestimation result of the deterioration risk.
 21. The image processingapparatus according to claim 20, wherein the processor changes a displaycolor tone of the notification form of the lesioned part candidate basedon the estimation result of the deterioration risk.
 22. The imageprocessing apparatus according to claim 20, wherein the processoranalyzes a nature of the lesioned part candidate and changes a displaydetermination level of display and non-display of the lesioned partcandidate based on the nature of the lesioned part candidate and theestimation result of the deterioration risk.
 23. The image processingapparatus according to claim 1, wherein the processor displays, on adisplay, the observation image and a detection marker indicating thelesioned part candidate.
 24. A non-transitory computer-readablerecording medium recording an image processing program, the imageprocessing program being for causing a computer to execute processingfor: receiving an observation image of a subject or the observationimage and system information; detecting a lesioned part candidate fromthe observation image; estimating a deterioration risk of endoscopyquality from the observation image or the system information;controlling a notification form of the lesioned part candidate from anestimation result of the deterioration risk of the endoscopy quality;and notifying the lesioned part candidate according to the control ofthe notification form.
 25. The recording medium according to claim 24,wherein the processing for estimating the deterioration risk of theendoscopy quality includes processing for analyzing a fatigue degree ofa user and calculating the deterioration risk of the endoscopy qualitybased on the fatigue degree.
 26. A diagnosis supporting methodcomprising: detecting a lesioned part candidate from an observationimage of a subject; estimating a deterioration risk of endoscopy qualityfrom the observation image or system information; and notifying thelesioned part candidate in a notification form corresponding to anestimation result of the deterioration risk of the endoscopy quality.27. The diagnosis supporting method according to claim 26, wherein theestimating the deterioration risk of the endoscopy quality includesanalyzing a fatigue degree of a user and calculating the deteriorationrisk of the endoscopy quality based on the fatigue degree.